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Abstract 

The main subject of the paper is everywhere complex sequences. An 
everywhere complex sequence is a sequence that does not contain substrings 
of Kolmogorov complexity less than an — 0{l) where n is the length of 
substring and a is a constant between and 1 . 

First, we prove that no randomized algorithm can produce everywhere 
complex sequence with positive probability. 

On the other hand, for weaker notions of everywhere complex sequences 
the situation is different. For example, there is a probabilistic algorithm that 
produces (with probability 1) sequences whose substrings of length n have 
complexity — 0(1). 

Finally, one may replace the complexity of a substring (in the defini- 
tion of everywhere complex sequence) by its conditional complexity when 
the position is given. This gives a stronger notion of everywhere complex 
sequence, and no randomized algorighm can produce (with positive proba- 
bility) such a sequence even if an is replaced by y/ii, log* n or any other 
monotone unbounded computable function. 



1 Introduction 

The paper considers binary sequences with substrings of high Kolmogorov com- 
plexity. Kolmogorov complexity of a binary string is the minimal length of a pro- 
gram that produces this string. We refer the reader to ID or 121 for the definition 
and basic properties of Kolmogorov complexity. 

The Levin-Schnorr Theorem (see, e.g., fP\) characterizes randomness of a 
sequence in terms of complexity of its prefixes. It implies that a «-bit prefix 
of a Martin-Lof random sequence has complexity n — 0{l). (Technically, we 
should consider monotone or prefix complexity here; for plain complexity we have 
n — C?(log?i) bound, but in this paper logarithmic precision is enough.) So se- 
quences with complex prefixes exists (and, moreover, fair coin tossing produces 
such a sequence with probability 1). 
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If we require that all substrings (not only prefixes) are complex, the situation 
changes. Random sequences no more have this property, since every random se- 
quence contains arbitrarily long groups of consequtive zeros (and these groups have 
very small complexity). 

However, the sequences with this property ("everywhere complex") still exist. 
The following Lemma (proved by Levin, see (3)) says that there exists a sequence 
where every substring has high complexity (though the condition is now weaker, 
the complexity is greater than an — 0(1) where n is the length and < a < 1). 

Here is the exact statement. We denote by 0){[ij)) a substring ft);G),+iG),+2 • • • (Oj-i 
of a sequence co; by K{u) we denote Kolmogorov complexity of a binary string u. 

Lemma 1 (Levin). Let a be a real number, < a < 1. There exists a sequence (0 
such that 

K{(o{[k,k + n))) > an-0{\). 
for all natural numbers k and n. 

Here the constant 0(1) may depend on a but not on n and k. 

Levin's proof in [3] used complexity arguments: informally, we construct the 
sequence from left to right adding bit blocks; each new block should increase the 
complexity as much as possible. 

Later it became clear that this lemma has combinatorial meaning: if for every 
n some 2"" strings of length n are "forbidden", there exists an infinite sequence 
without long forbidden substrings. This combinatorial interpretaion shows that the 
statement of the lemma (and even a stronger statement about subsequences, not 
only substrings) is a corollary of the Lovasz local lemma (see [T, 5 |). Recently two 
more proofs were suggested (by Joseph Miller |6| and Andrej Muchnik). 

Before stating our results, let us mention the following slightly generalized 
version of Levin's lemma. Though not stated explicitly in lO, it can be proved by 
the same argument. 

Lemma 2 (Levin, generalized). Let a be a real number, < a < 1. Then there 
exists a sequence (0 such that 

K{(0{[k,k + n)) I k,n) > an-0{\). 

for all integers k, n. 

Here K{x\y) denotes conditional Kolmogorov complexity of a string x when y is 
given (i.e., the minimal length of a program that transforms y to a). The difference 
is that substrings are now complex with respect to their position and length (so, for 
example, the binary representation of n can not appear starting from position n). 
In combinatorial terms, we have different sets of forbidden substrings for different 
positions. 

One can ask how "constructive" could be the proofs of Levin's lemma and 
its variants. There are several different versions of this question. One may as- 
sume that the set of forbidden strings is decidable and ask whether there exists a 
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computable sequence that avoids all sufficiently long forbidden strings. Miller's 
argument shows that this is indeed the case, though a similar question of 2D con- 
figurations (instead of ID sequences, cf. |4]) is still open. 

In this paper we consider a different version of this question and ask whether 
there exists a probabilistic algorithm that produces a sequence satisfying the state- 
ment of Levin's Lemma (or some version of it) with positive probability. 

2 The results 

We say that a sequence co is a-everywhere complex if 

liL{(o{[k,k + n))) > an-c 
for some constant c and for all integers k and n. 

Theorem 1. No probabilistic algorithm can produce with positive probability a 
sequence (0 that is a-everywhere complex for some a £ (0, 1). 

Theorem 2. Let YT=o computable converging series of nonnegative rational 

numbers. There exists a probabilistic algorithm that produces with probability 1 
some sequence ft) such that 

K(ftj([^,^ + «))) >a[iog„]«-c 

for some c and for all k and n. 

Theorem 3. No probabilistic algorithm can produce with positive probability a 
sequence (0 with the following property: there exists a non-decreasing unbounded 
computable function g : N — N such that 

}L{(o{[k,k + n)) I k,n) > g(n) 

for all k and n. 

Theorem [T] and [2] complement each other: the first one says that a-everywhere 
complex sequences for a fixed a > (even very small) cannot be obtained by a 
probabilistic algorithm; the second one says that if we allow sublinear growth and 
replace the bound an by ^/n or n/log^n, then the probabilistic algorithm exists. 
(There are intermediate cases where none of these theorems is applicable, say, 
?i/log?i bound; we do not know the answer for these cases.) 

Theorem [3] says that Theorem [2] cannot be extended to the case of generalized 
Levin's lemma; here the answer is negative for any computable non-decreasing 
unbounded function. 
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3 Proof of Theorem |2] 



Let us start with the positive result. 

Proof of theorem^ The idea of the construction is simple. We fix some com- 
putable function / : N — > N and then let ft),- = Tj(,) where T,- is a sequence of ran- 
dom bits (recall that we construct a probabilistic algorithm that uses random bit 
generator). 

In other words, we repeat the same random bit Xj several times at the locations 
(Oi where /(/) = j. Why does this help? It allows us to convert bounds for the 
complexity of prefixes of T into bounds for the complexity of substrings of (O. 
Indeed, if we have some substring of (O and some additional information that tell 
us where several first bits of T are located in the substring, we can reconstruct a 
prefix of T. 

More details. We may assume without loss of generality that n, the length of 
a substring, is large enough. We may also assume that « is a power of 2, i.e., 
that n = 2'" for some m. Indeed, for every substring x we can consider its prefix 
x' whose length is the maximal power of 2 not exceeding the length of x. The 
bound for complexity of x' gives the same (up to a constant factor) bound for the 
complexity of x. 

Consider the substring (o{[k,k + 2'^)) for some k and m. We want it to contain 
all the bits from some prefix of T, more specifically, the first amT" bits Tq, . . . , Tq„,2'"-i 
of T. (We may assume without loss of generality that am2"^ is an integer.) 

To achieve this, we put each of these bits at the positions that form an arithmetic 
progression with common difference 2'". The first term of this progression will be 
smaller than its difference, and therefore each interval of length 2™ contains exactly 
one term of this progression. 

In this way for a given m we occupy a„,-fraction of the entire space of indices 
(each progression has density 1 /2"' and we have (3^2'" of them). So to have enough 
room for all m we need that < 1- This may be not the case at first, but we can 
start with large enough mo to make the tail small. 

Technically, first we let m = rriQ and split N into 2"' arithmetic progressions 
with difference 2™. (First progression is formed by multiples of 2"', the second 
is formed by numbers that are equal to 1 modulo 2™, etc.) We use first a„j2'" 
of them for level m reserving the rest for higher levels. Then we switch to level 
m = mo + 1, splitting each remaining progression into two (even and odd terms), 
use some of them for level mo + 1, convert the rest into progressions with twice 
bigger difference for level mo + 2, etc. (Note that if in a progression the first term 
is less than its difference, the same is true for its two halves.) 

This process continues indefinitely, since we assume that +amo+\ + ...<!. 
Note that even if this sum is strictly less than 1 , all natural numbers will be included 
in some of the progressions: indeed, at each step we cover the least uncovered yet 
number So we have described a total computable function / (its construction 
depends on mo, see below). 
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Now we translate lower bounds for complexity of prefixes of T into bounds for 
complexity of substrings of ft): the substring 0){[k,k + 2'")) contains first a„,2'" bits 
of T (for m > mo), and the positions of these bits can be reconstructed if we know 
k mod I'" and the function /. This additional information uses 0{m) bits (recall 
that mo < m and it determines /). So 

K(co([A:,)t + 2'"))) >K(T([0,a,„2"')))-O(m) >a^2'"-0(m). 

The last term 0{m) can be eliminated: increasing a„, by 0{m) /2"', and even more, 
say, by m^/2'", we do not affect the convergence. (The bounds presented are lit- 
erally true for prefix complexity; plain complexity of prefixes of T is a bit smaller 
but the difference again can be easily absorbed by a constant factor that does not 
affect the convergence.) □ 

4 Proof of Theorem |3] 

The proofs of Theorem [T] and Theorem [3] are based on the same idea. We start with 
proving Theorem [3] as it is simpler. 

Proof of Theorem 3. Fix some probabilistic algorithm A. We need to prove that 
some property ("there exists function g such that K{o){[k,k + n))\k,n) > g{n) for 
all k and n") has probability with respect to the output distibution of A. Since 
there are countable many computable functions g, it is enough to show that for a 
given g this happens with probability 0. So we assume that both A (probabilistic 
algorithm) and g (a computable monotone unbounded function) are fixed, and for 
a given £ > prove that the property "K{o){[k,k + n))\k,n) > g{n) for all k and n" 
has probability smaller than e. 

Assume first that probabilistic algorithm A produce an infinite output sequence 
with probability 1 , and therefore defines a computable probability distribution Pa 
on the Cantor space of infinite sequences. 

Consider some n. First we prove that for large enough N it is possible to select 
one "forbidden" string of length n for each starting position ^ = 0, 1, . . . jA'^ — 1 in 
such a way that the event "output sequence avoids all the forbidden strings" (at the 
corresponding positions) has probability less than e. 

It can be shows in different ways. For example, we can use the following prob- 
abilistic argument. Let us choose the forbidden strings randomly (independently 
with the random bits used by A). For every output sequnce of A the probability that 
it avoids all randomnly selected "forbidden" strings is (1 — 2^")^ which is less than 
e if A/^ is sufficiently large. Therefore, the overall probability of the event "output 
of A avoids all forbidden strings" (with respect to the product distribution) is less 
than e. Now we use averaging in different order and conclude that there exists one 
sequence of forbidden strings with the required property. 

After the existence of such a sequence is proved, it can be found by exhaustive 
search (recall that Pa is computable). Let us agree that we use the first sequence 
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with this property (in some search order) and estimate the complexity of forbid- 
den strings when length n and position k are known. The value of A/^ is a simple 
function of n and e (which is fixed for now, as well as A), and we do not need any 
other information to construct forbidden strings. So their conditional complexity 
is bounded and is less than g{n) for large enough n. So the probability that all the 
substrings in the output of A will have complexity greater than g(their length), is 
less than e. 

It remains to explain how to modify this argument for a general case, with- 
outh the assumption that A generates infinite sequences with probability 1. Let 
us modify the function N{£,n) in such a way that (1 -2-")'^("-'^) < e/2. Con- 
sider the probability of the event "A generates a sequence of length N{n,e) + n". 
If somebody gives us (in addition to n and £) an approximation from below for 
this probability with error at most e/2, we may enumerate A's output distribution 
on strings of length N + n and stop when the lower bound is reached. Then we 
apply the argument above using this restricted distribution and show that for this 
restricted distribution the probability to avoid simple strings is less than e/2, which 
gives £-bound for the full distribution (since they differ at most by s/2). It is im- 
portant here that the missing information is of size log(l/£) + 0(1), so for a fixed 
e we need 0(1) additional bits. □ 

5 Proof of Theorem [1] 

The proof of Theorem [T] is similar to the preceding one, but more technically in- 
volved. In the previous argument we were allowed to choose different forbidden 
strings for different positions, and it was enough to use one forbidden string for 
each position. Now we use the same set of forbidden strings for all positions, and 
the simple bound (1 — 2^")^ is replaced by the following lemma. 

Lemma 3. Let a € (0, 1). For every £ > there exist natural numbers n and N 
{with n <N) and random variables s^n^^n+x , whose values are subsets of 
B",B"+\ . . . respectively, that have the following properties: 

(1) the size of subset s^i never exceeds 2"'; 

(2) for every binary string x of length N the probability of the event 'for some 
i G {n, . . . ,N} some element of^/j is a substring ofx" exceeds 1 — £. 

The number n can be chosen arbitrarily large. 

(We again use the probabilistic argument; this lemma estimates the probability 
for every specific x and some auxiliary probability distribution; the output distribu- 
tion of randomized algorithm A is not mentioned at all. Then we use this lemma to 
get an estimate for the combined distribution, and change the order of averaging to 
prove the existence of finite sets A„, . . . ,A/v with required properties.) 

Proof. First let us consider the case a > 1/2. Then we actually need only two 
lengths n and N, where N ^n,di\\ other lengths are not used and the corresponding 
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random subsets can be empty. For length n, we consider a uniform distribution on 
all sets of size 2""; all these sets have equal probabilities to be a value of random 
variable For length the set si/^ is chosen in some fixed way (no randomness), 
see below. 

Assume that some string x of length is fixed. There are two possibiUties: 

(a) there are at least 2"/^ different substrings of length n in x; 

(b) there are less than 2"/^ different substrings. 

In the first case (a) strings of length n play the main role. Let 5 be a set of 
n-bit strings that appear in x; it contains at least 2"/^ strings. The probability that 
the desired event does not happen does not exceed the probability of the following 
event: "making 2"" random choices among n-bit strings, we never get into 5"'. 
(It is a bit smaller, since now we can choose the same string several times.) This 
probabiUty equals 

(1 _2"/2)2"" = _2«/2-)2"/22(«-V2)" ^ ^^^^^2(«-i/2)" 

and converges to zero (rather fast) as « -> oo. 

In the second case (b) strings of length N come into play. We may assume that 
A'^ is a multiple of n. Let us split x into blocks of size n. We know that x has some 
special property: there are at most 2"/^ different blocks. Note that for large the 
number of strings with this special property is less than 2"". Indeed, to encode 
such a string x, we first list all the blocks that appear in x (this is a very long list, 
but its length is determined by n and does not depend on A'^), and then specify each 
block by its number in this Ust. In this way we need N /2 + 0{\) bits (the number is 
twice shorter than the block itself) and this is less than aA'^ for large A^. So for such 
a large A^^ we may include all strings with this property in Aj^ and get the desired 
effect with probabiUty 1. 

Now let us consider the case when a > 1/3 (but can be less than 1/2). Now 
we need three lengths n\ <C W2 *C "3- We will use that is a multiple of «i, and 
?i3 that is a multiple of /i2- For length n\ we again consider a random set of 2*^"' 
strings of length «i. It guarantees success if the string x contains at least 2(^/^'"i 
different blocks of length n\ . 

Now we compile a list of possible blocks of size n2 that are "simple", i.e., 
contain at most 2(^/-''"' different blocks of size ni. The same argument as before 
shows that a simple block can be described by (2/3)^2 + 0{\) bits, where 0(1) 
depends only on ni. Now A„2 is a random set of 2""^ simple blocks of size n2- 
Then the argument again sphts into two subcases. (Recall that we assume now that 
X is made of simple blocks of size «2-) 

The first case happens when x contains more than 2"2/3 different simple blocks. 
Then with high probability some block of x appears in A„2- 

The second case happen when x contains less than 2"2/3 different simple blocks. 
Then x can be encoded by the list of these blocks, and this requires n^/d + 0(1) 
bits. So if ns is large enough (compared to n2), all possibilities can be included in 
A„3, and this finishes the argument for a > 1/3. 



7 



Similar argument with four layers works for a > 1/4, etc. 



□ 



This lemma will be the main technical tool in the proof of Theorem [T] But first 
let us prove a purely probabilitic counterpart of Theorem [T] that is of independent 
interest. 

Theorem 4. Let a € (0, 1). For every probability distribution P on Cantor space 
Q., there exist sets Ai,A2, . . . of binary strings such that 

(1) the set An contains at most 2"" strings of length n; 

(2) with P-probability 1 a random sequence has substrings in Ai for infinitely 
many i. 

The possible "philosophical" interpretation of this theorem: one cannot prove 
the existence of sequences that avoid almost all A, by a direct application of the 
probabilistic method, something more delicate (e.g., Lovasz local lemma) is needed. 

Proof. Let us first consider sequences of some finite length A'^ and the induced 
probability distribution on them. We claim that for every e and for large enough 
we can choose Ai, . . . ,Aa? in such a way that they satisfy (1) and P-probability to 
avoid them is less than e. 

To show this, consider (independent) random distribution on strings of lengths 
l,...,N provided by the lemma. What is the probability that random string avoids 
random set (with respect to the product distribution of P and the distribution pro- 
vided by the lemma)? Since for every fixed string the probability is less than e 
(assuming N is large enough), the overall probability (the average) is less than e. 
Changing the order of averaging, we see that for some Ai , . . . ,A/v the corresponding 
P-probability is less than e. 

Note that in fact we do not need short strings; strings longer than any given 
n are enough (if N is large). So we can use this argument repeatedly with non- 
overlapping intervals {ni,Ni) and £,■ decreasing fast (e.g., = 2^')- Then for P- 
almost every sequence we get infinitely many violations. Moreover, since the series 
£i is converging, P-almost every sequence hits all but finitely many of A, (Borel- 
CantelU lemma). □ 

Now we are ready to prove the weak version of Theorem [T] 

Let a G (0,1). There is no randomized algorithm that produces a- 
everywhere complex sequences with probability 1. 

(The difference with the full version is that here we have probability 1 instead 
of any positive probability and that the value of a is fixed.) 

To prove this statement, let us consider the output distribution P of this algo- 
rithm. Since the algorithm produces an infinite sequence with probability 1, this 
distribution is a computable probability distribution on the Cantor space. This mea- 
sure can be then used to effectively find sequences A^,- and sets A,- as described 
so that with P-probability 1 a random sequence hits all A; except for finitely many 
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of them. Since the sets A, can be effectively computed and have at most 2"' ele- 
ments, every element of A,- has complexity at most ai + C?(log/); the logarithmic 
term can be absorbed by a change in a. 

This argument shows also that for every computable probability distribution P 
and every a G (0, 1) there exists a Martin-Lof random sequence with respect to P 
that is not a-every where complex. One more corollary: for every a G (0, 1) the 
(Medvedev-style) mass problem "produce an a-everywhere complex sequence" is 
not Medvedev (uniformly) reducible to the problem "produce a Martin-Lof random 
sequence". 

It remains to make the last step to get the proof of Theorem [T] 

Proof of TheoremUl If the probability to get an sequence that is a-everywhere 
complex is positive, then for some a the probability to get a a-everywhere complex 
sequence for this specific a is positive. (Indeed, we may consider only rational a 
and use countable additivitey.) 

So we assume that some a is fixed and some probabilistic algorithm produces 
a-everywhere complex sequences with positive probability. We cannot apply the 
same argument as above. The problem is that the output of the algorithm (restricted 
to the first N bits) is a distribution on that is not computable (the probability 
that at least N bits appear at the output, is only a lower semicomputable real). 
However, for applying our construction for some it is enough to know the output 
distribution up to precision £,/2 (in terms of statistical distance), as explained in 
the proof of Theorem [3l we replace our distribution by its part, and the error is at 
most e/2. For this we need only log(l/£,) + 0(1) bits of advice, which can be 
made small compared to an. □ 

Now we get a stronger statements for mass problems: 

Theorem 5. The mass problem "produce an everywhere complex sequence " is not 
Muchnik {non- uniformly) reducible to the problem "produce a Martin-Lof random 
sequence ". 

Proof. Indeed, imagine that for every random sequence there is some oracle ma- 
chine that transforms it to an everywhere complex sequence. Since the set of oracle 
machines is countable, some of then should work for a set of random sequences that 
has positive measure, which contradicts Theorem [T] □ 

The author thanks Steven Simpson for asking the question, and Joseph Miller 
and Mushfeq Khan for the discussion and useful remarks. 
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